Response to “Methodological issues in the designing and reporting of frequentist and Bayesian meta‐analysis to assess COVID‐19 outcomes among PLHIV with various comorbidities” by Ram Bajpai, Vivek Verma and Gyan Prakash Singh

Response to “Methodological issues in the designing and reporting of frequentist and Bayesian meta-analysis to assess COVID-19 outcomes among PLHIV with various comorbidities” by Ram Bajpai, Vivek Verma and Gyan Prakash Singh Haoyi Wang and Kai J. Jonas§ §Corresponding author: Kai J. Jonas, Department of Work and Social Psychology, Maastricht University, Maastricht 6200ER, The Netherlands. Tel: +31 43 38 84068. (kai.jonas@maastrichtuniversity.nl)

Our colleagues Bajpai et al. have submitted a Letter to the Editor regarding our recently published meta-analysis "The likelihood of severe COVID-19 outcomes among PLHIV with various comorbidities: a comparative frequentist and Bayesian meta-analysis approach [1]." We appreciate their feedback and request for clarification and would like to respond to their letter.
First, we have indeed used the Preferred Reporting Items for Systematic reviews and Meta-Analyses 2020 in our flow diagram. Unfortunately, we cited the wrong resource for this. We apologize for this error, yet we deem the impact to be rather small as the flow diagram is correct. We did not register the meta-analysis with PROSPERO as the registration procedure currently takes more than 4 months. The urgency and timely relevance of the analysis did not allow for a registration, from our point of view. In times of crisis, a value weighing is necessary between adhering to timeintensive procedures or contribute to directly relevant care provision.
Second, the authors suggest that there is missing information about the model selection (i.e. fixed or random). We disagree with this view. A random-effects model assumes that each study estimates a different underlying true effect, and these effects have a distribution (usually a normal distribution). A fixed-effects model should be used only if it is reasonable to assume that all studies share the same, common effect [2]. If it is not reasonable to assume that there is one common effect size, then the random-effects model should be used. If the studies are heterogeneous from a clinical and methodological point of view, it is unreasonable to assume that they share a common effect [2]. In our study, given the heterogeneous studies included as well as their purposes (different comorbidities, different settings and different study designs), it is not reasonable to use fixed-effect model. We believe it was not necessary to further motivate our selection of a random effects model as the default, and we consider this such a claim irrelevant for this paper, as this is not a methods paper.
Third, the authors ask us to present the results with the prediction interval. This is a recent recommendation that is not yet standard practice. Please find the prediction interval reported in Table 1.
Fourth, the authors believe that our publication bias assessment is prone to chance bias. We disagree with this view. Indeed, the Cochrane Collaboration recommends that tests for funnel plot asymmetry should be used only when there are at least 10 studies included in the meta-analysis, because when there are fewer studies, the power of the tests is too low to distinguish chance from real asymmetry. But we did not detect any asymmetry, hence, the chance bias does not apply here. Furthermore, Tang and Liu recommend at least six studies for studying publication bias; we included nine [3].
Fifth, the authors would have preferred more in-depth information about the Bayesian approach and its findings. We used the Bayesian approach to provide a more robust estimate, also given the small number of studies. The similarity of effect estimates does not speak against the approach as such. We agree that the methods are novel to some readers, yet we provide all necessary steps and references in the paper for an interested reader to replicate the procedure in this or other contexts.
Sixth, the authors suggest to use the posterior predictive probability to assess the performance of the prior. We disagree with this view. In the paper referred to by the authors, Lewis et al. stressed the fact that their conditional predictive ordinates approach [4], compared to methods described Note: Hospitalization includes hospitalized but not requiring supplemental oxygen, hospitalized but requiring supplemental oxygen, hospitalized with non-invasive ventilation, hospitalized on invasive mechanical ventilation or ECMO and mortality; severe outcome was defined as fever or suspected respiratory infection plus respiratory rate greater than 30 breaths per min, oxygen saturation of 93% or less on room air, or acute severe respiratory distress (acute lung infiltrate in chest imaging and ratio of partial pressure of arterial oxygen to fractional concentration of oxygen in inspired air [PaO2/FiO2] of ≤300). Critically ill individuals were those with rapid disease progression and respiratory failure with need for mechanical ventilation or organ failure that needs monitoring in an intensive care unit. The scale on the effects is log odds ratio. Abbreviations: 95% CI, 95% confidence interval; 95% CrI, 95% credible interval.
a Did not report type of hospitalization. b Half-normal distributions with scale 0.5.
by Gelfand and Ghosh [5], only evaluates the posterior predictive probability density but does not simulate from it, and both methods relied on applying posterior predictive distribution. Also, in our sensitivity analysis, we checked the prior predictive cumulative distribution function (CDF, as stated in Figure S5 of the Supporting Information of [1]). The CDF is a probability, and the probability density function (PDF) is the derivative of the CDF, and the predictive distribution function is a probability as well. Also, in our view, prior predictive distribution can be understood as a concept in two ways: a PDF in case of a continuous random variable and a probability mass function for discrete values of the random variable. We thus disagree with the authors' view based on Lewis' statement. We agree with the view that Bayes factors are useful for guiding an evolutional model-building process, as stated in the paper from Kass and Raftery [6]. However, it is not the only method for model selection. As evaluated by Lynch and Western [7], the posterior predictive distribution can be compared to the observed data to assess model fit, and the posterior predictive distribution provides a useful set of statistics for assessing model fit.
Finally, the authors would like to see more elaboration on the inferential procedure presented in the manuscript. We agree that Bayesian methods commonly are computationally more demanding than other methods, usually, these require the determination of high-dimensional integrals, in most cases, with Markov chain Monte Carlo (MCMC) methods. However, in the present case of random-effects meta-analysis, where only two unknown parameters are to be inferred, computations may be simplified by utilizing numerical integration or importance resampling [8]. According to Röver and Friede, computations may be done partly analytically and partly numerically, offerring another approach to simplify calculations via the DIRECT algorithm, which is implemented in the bayesmeta R package [9,10]. Hence, using the bayesmeta R package, direct access to quasi-analytical posterior distributions can be provided, without having to worry about setup, diagnosis or post-processing of MCMC algorithms. Also, inference based on MCMC output always contains a certain noise component due to the finite number of samples, which may sometimes constitute a nuisance. Use of the bayesmeta package instantly provides accurate posterior summary figures analogous to output familiar from frequentist metaanalysis output [10]. Posterior distributions may be accessed in quasi-analytical form, and advanced methods, for example for prediction or shrinkage estimation, are also provided. Computations are fast and reproducible, allowing for quick sensitivity checks and facilitating larger-scale simulations. Bayesmeta implementation yields consistent results through calibration checks [10]. Using bayesmeta in R, we thus believe these aspects are not relevant, also given the fact that this is not a methods paper.

A U T H O R S ' A F F I L I AT I O N S
Department of Work and Social Psychology, Maastricht University, Maastricht, The Netherlands

C O M P E T I N G I N T E R E S T S
The authors have no competing interests to report.

A U T H O R S ' C O N T R I B U T I O N S
HW and KJ equally contributed to drafting the response and both have read and approved the final manuscript.

A C K N O W L E D G E M E N T S
None.